Module 13 - Deep Learning

Overview

During this week, we continue to learn about neural networks, moving from the basics of individual neurons to challenges that arise in deep architectures. We introduce recurrent neural networks, deep neural networks, and discuss some of the challenges with training deep neural networks. We explain backpropagation, and show how to use ReLU layers to mitigate vanishing and/or exploding gradients, and talk about hyperparameter optimization in stochastic gradient descent.

Lab 7 is due at the end of the week.

Learning Objectives

RNN Architectures
Challenges with Deep Learning: Vanishing and Exploding Gradients
ReLU Layers
Hyperparameter Choices for Stochastic Gradient Descent

Readings

ISLP (Introduction to Statistical Learning): 10.5-10.8

Overview

Learning Objectives

Readings

Videos